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Method for Stereoscopic Measuring Image Points and Device for Carrying 
Out Said Method 

The invention relates to stereometry, particularly to non-contact methods of object spatial 
characteristics determination by it's stereoscopic images and can be used in photogrammetry, 
medicine, construction, architecture, biology, systems of object identification, natural sources 
research, assessing risks of natural and man-caused disasters and the effects thereof and for other 
purposes. 

There is a known method of stereoscopic measuring image points, which consists of a 
stereoscopic measuring stereo-model by determining the position of the aiming axis of eyes, 
relative to the main optical axis of the monitoring system and tracking the monitoring results by 
determining the position of the aiming axis of eyes, which is the closest to the one claimed 
herein (SUNo. 551504, G01C 11/04, 25.03.1977). 



The weak points of said method are: 

Absence of the visual control of the stereo-images monitoring process, and, therefore, 
low measuring productivity. 

Low precision of determining the position of the aiming axis of eyes at the moment of 
eye focusing. 

Eye fixation is determined by the physiological features of the human vision; it is a 
transitional process of a sight fixation at the time of focusing on any point of the subject during 
certain period of time. The said method implies the detection of the focus moments on the base 
of the amplitude analysis and of position speed change of the aiming axis of eyes, but because 
the human eye continuously makes micro-movements, sight fixation is not a geometric point 
which is stable in time and space, but a zone of undetected shape, which is being formed in some 
range of time. Since the concrete points with the discrete data are required for the 
photogrammetric construction, the use of fixations determined by the method described, does not 
allow detecting the position of the aiming axis of eyes with the required for the photogrammetric 
measuring precision. 

On the other hand, because of the individual lateral particularities, the movements of the left 
and right eye are not synchronic, that is why the selected fixations do not determine the moment, 
when both eyes look at the same point of the subject. 

The principal scheme of the device (SU No. 551504, G01C 1 1/04. 25/03/1977), which allows 
carrying out the method of stereoscopic measuring the image points, is already known. 

The described principal scheme of the device consists of the photogrammetric device of the 
analytic type, containing the processor inside; with the built-in system of image entry into the 
television automates and eye movements' electronic analyzers on the base of vidicons. 

The device is assigned to measure the photographic images using the prism-cube with partly 
silver-plated internal edge for the construction of the observer's eye image in the television 
automate vidicon ocular. 
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The weak points of the principal scheme of the described device are: 

- lack of the "feedback" system (i.e. reflection of the measuring data on the images 
themselves), which leads to lack of observation control. 

- lack of precise compensation mechanism of the head movement, because the 
compensation offered in the head movement device is based on the measuring of an eye 
glare, which is formed by infrared sources of radiation and does not take into 
consideration the geometrically uneven shape of the eye, which causes nonlinearity of 
change of the glare position at the time of eye movement; besides, while the head 
position is changing at the time of focusing on the point of the image, the eye makes 
some compensating movements, which also leads to the nonlinear modification of the 
glare position; 

impossibility of the precise discrete fixation of the measuring data with the use of the 
systems, which are built on the base of the television automates with the analog vidicons; 
low precision of the detection of the eye center pupil, which is caused by low contrast of 
the human eye images, if the light sources assigned only for images highlighting are 
used; 

- losses of the optical radiation at the time of passing through the prism-cube with the 
partially silver-plated internal edge; 

- small range of vision of the optical systems, which are used in analog photogrammetric 
devices. 

There is a device of the precise detection of the sight direction (A Precise Eye-Gaze 
Detection and Tracking System, A.Perez, M.L.Cordoba, A. Garcia, R. Mendez, MX. 
Munoz, JL. Pedraza, F. Sanchez, WSCG'2003, February 3-7, 2003 Plzen, Czech Republic), 
carrying out the method of Eye-Gaze Detection, which is the closest to claimed herein. 

The described device consists of the surveillance camera for tracking eye movements, 
panoramic camera for tracking head movements, system for image entry into personal 
computer, and four infrared radiators for forming special glare-marks on the eye surface. 

The weak points of the described device are lack of precise compensation of the head 
movements and necessity of use the video-cameras with a very high definition. Movements 
of the head are detected in the device by the images of observer's face (in particular, by 
identification of the eyes on the entire face image), which is done by the wide-angle video- 
camera for tracking the head. Parts of the face have some relative movements, that is why 
they can not be used as the stable basic points of the head, and that is why the method used in 
the device is not strict and, therefore, cannot comply with the requirements for high-precision 
measuring. The device presumes the use of only one video-camera to receive the eye images, 
but because the eyes of observer are located at some distance from each other, in addition to 
the images of the eyes there is insignificant information data, capturing by the camera (image 
of the part of the face in the bridge of nose area). Therefore, in order to receive precise 
enough images, it is necessary to increase the requirements for the camera's definition 
capacity, i.e. to use matrix of large dimension in the video-cameras, but increase of the 
matrix dimension leads to increase of the volume of the incoming video-information and 
noticeably increasing the requirements for the productivity and speedy action of the video- 
capture plate. The invention solves the problem of increasing the productivity of measuring 
the spatial characteristic of the object by its images on the stereoscopic pictures. 

The problem can be solved by the following: according to the invention in the method of 
stereoscopic measuring image points, including construction of a stereoscopic model based 
on two overlapping images, detection of the position of the aiming axis of eyes in 
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stereoscopic perception of that model and recording the observation results at eye fixation 
moments, the projection of the sight fixation area on the monitor screen with the images 
observed is computed and the typical points of the observed object, corresponding with those 
areas, on the fragments of digital stereo-images, are selected. 

There are additional choices to carry out the method, for which it is reasonable: 

- to identify typical points of the same name of the observed subject, which are selected on 
the fragments of the digital stereo-images, correlating with areas of sight fixation, for the 
left and right eye by time synchronization; 

- To identify typical points of the same name of the observed subject, which are selected 
on the fragments of the digital stereo-images, correlating with areas of the sight fixation, 
for the left and right eye, starting with the condition of crossing of the corresponding 
rays, determined by the vectors coplanarity equation; 

to do the calibration of the system before starting the observation, by observation of the 
image with test-objects with the known position data in the system of the position data of 
the main monitor, comparing the position data of the pupils of the eyes, determined in the 
system of position data of the video-camera, with the position data of the test objects, 
shown on the main monitor, and the subsequent mathematic dependencies, describing 
mutual transformations of position data. 

- At the time of system calibration, to position the test objects for observation in different 
conditions (for example, time, duration and order of appearance of the test objects, 
disposition, size, shape and color of the test objects, surrounding background, static or 
dynamic conditions of the test object appearance); 

- while observing, to visually control the measuring data on the screen of the main monitor 
by imprinting the color markers into the image area, coordinating with that fixation; 

- To do a visual control of measuring on the main monitor screen by modifying the color 
parameters of the area on the stereo-image, corresponding with that fixation; 

To do the compensation of the observer's head movements by computing the movement 
in the position of the aiming axis of the eyes by images of certain parts of the observer's 
head. 

- To do the compensation of the head movement of the observer by tracking several marks, 
fixed on the head of the observer; 

- To track the head movement by the marks fixed close to eyes in a way to get the images 
of those marks captured by the video-cameras, recording the observer's eyes movements; 

- To make the marks for tracking the observer's eye movements in a special (for example, 
ellipsoid) shape, what allows detecting precisely the position and orientation of the mark, 
and, accordingly, movements of the observer's head. 

- To detect the position data of the motion of the head in two mutually perpendicular 
planes; 

- To detect the pupil of the eye position while recording the eyes movements in three- 
dimensional space by receiving two images of each eye by two synchronized video- 
cameras, fixed with different foreshortening. 

The problem can be solved by the additional input, according to the invention, the 
construction made in the shape of an eyeglasses frame with the specially shaped marks, 
positioned on vertical plane, into the content of the device for stereoscopic measuring image 
points, consisting of two video-cameras for recording the eye movements, video-cameras for 
tracking the head movements, system of the image video-capture by the personal computer, 
monitor for displaying the Image, system of stereo-surveillance, allowing to observe 
stereoscopic images, in a way to get captured by the eye movements recording cameras 
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Additional versions of the device are possible, wherein it is reasonable to do the 
following: 

- To install the additional specifically shaped marks, located on the horizontal plane, into 
the eyeglasses frame and to install a mirror, placed above the observer's head, into the 
device; in which connection to install the video-camera for tracking the head movements 
in a special way for capturing at the same time the part of the head and the reflection in 
the eyeglasses frame mirror with the specially shaped marks placed on the horizontal 
plane on it ; 

- To install in the system, in addition to the main two video-cameras for tracking the 
movements of each eye separately, two additional video-cameras, placed in a way to 
synchronically record the movements of each eye by the main and the additional video- 
cameras from two points. 

- To install an additional monitor for visual tracking the observation and operation of the 
observation process; 

To install the system of the infrared highlighting of the eyes area; 

To install infrared color filters on the cameras to cut off the parasite highlighting in the 
visible range of spectrum; 

The mentioned above advantages and also the particularities of that invention explained 
by the best versions of it's carrying-out with the references to the enclosed figures: 

Figure No.l - typical trajectories of the eye at the time of sight fixation while focusing on the 

point of the object; 

Figure No. 2 - scheme of the stereoscopic observation of the stereo-Images on the monitor 
screen; 

Fig. No. 3 and Fig. No. 4 - general view of the device for measuring the three-dimensional 
position data based on its stereo-images; 

Fig. No. 5 - eyeglasses frame with the specifically shaped marks (for example, ellipsoidal 
shape), for the recording of the head movements; 

Fig. No. 6 - scheme of locations of the main and additional video-cameras for the recording 
of the eye movement in three-dimensional space. 

Construction of the three-dimensional model of the object in the real time while focusing 
on it's visual copy on two flat stereo-images can be done by tracking the observer's eyes 
micro-movements and recording the sight fixations with the subsequent computing the 
multitude of the points of intersection of the corresponding (paired) rays, determinating the 
homogenic virtual surface which is identical to the geometric surface of the object. 

The determination of the sight fixations can be done by dividing the basic consecution of 
the eye movement contents into areas of fast movement (saccades) and areas of the sight 
stabilization (fixations) separately for each eye. On the fig.l The typical trajectories of the 
eye are pictured at the time of the sight stabilization while focusing on the point of the object 

- areas of fixations 1 (marked with dotted lines). As a rule, detection of the point of fixation 
2 ( highlighted by the firm line) can be done by computing the simple geometric average or 
the average weighted centroid of the points of the sight trajectory in the limits of the fixation 
area 1 (fig.l), however, as it is shown on the fig.l, for that, the problem of vagueness of 
choice of the concrete points of the fixation 2 occurs, which is caused by significant 
dispersion of the points of trajectory of the eye movement in the limits of fixation area 1. 

It is suggested to solve that vagueness in the following way. Because the purpose of the 
stereoscopic measuring is to determine the spatial characteristics of the object, the observer 
focuses on the typical points of the object, making the object different from the environment 
and determining it's shape and size. It is naturally to assume, that the projection of the sight 
fixation area on the observed image on the monitor screen contains one or several those 
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points. Position data of those points on the digital image can be found automatically, 
applying the Harris algorithms, KLT or similar. 

In practice, the algorithms described can select several typical points on the image 
fragment, corresponding with that fixation area. Because a human being, physiologically, can 
not stabilize his/her sight on two different points of object, it is suggested to synchronize by 
time the fixation points, selected in the fixation areas of the left and right eye with the use of 
algorithms for searching the typical points on the image. Time synchronization allows to 
decrease ambiguity of the detection of the typical points on the image in the limits of the 
fixation area, corresponding to the focusing on the object by two eyes at the same time, 
however, because of the lateral particularities of the certain individual (dynamic asymmetry 
of one of the eyes, i.e. some "delay", "gap", which is similar to the "right-hander, left- 
hander" effect), that direction still can not comply with the actual state of the corresponding 
rays, pre-existing at the time of focusing on the concrete point of the object. 

That ambiguity is solved by the analysis of the geometric intersection of the aiming rays 
of the left and right eye at the time of stereoscopic focusing on the point of the object. 

The characteristics of the human binocular vision are such, that horizontal spatial parallax 
P between the pair of the corresponding points aj and a r on two images, located on the same 
plane ( with the condition of their separate observation by the eyes, fig.2) causes in human 
being the sensation of perception of the certain point, located out of the plane. As it is shown 
on the fig.2, the plane D is a display plane with the stereo-images, on which the observer's 
eyes are focusing, and axis R\ and R r are the vision Axis which are corresponding with the 
left L and right R eye, and B is an eyes base. While focusing separately on two 
corresponding points ai and ar, reflected on the left and right image of the stereo-pair, the 
image of the point A of the object's virtual model, formed as a result of intersection of the 
sight Axis R and R r , is formed in the human brain. The condition of the geometric 
intersection of the corresponding rays is determined by the condition of belonging of the 
vectors R and R r to the same plane, passing through the eyes base B. That condition is 
written by the vectors coplanarity equation: B (R x R r ) = 0 

Therefore, in the method suggested, the points-"candidates" 2, selected in the limits of 
the corresponding fixation areas for the left and right eyes, first have to be synchronized, and 
then have to be checked for compliance with the condition of vectors complanarity. The 
belongings of the corresponding vectors to the same primary plane is a strict geometric 
condition for observation of the certain point on the stereo-image, and, therefore, the 
multitude of the points of intersection of the corresponding (conjugate) rays, satisfying the 
condition of the complanarity, while focusing on the stereomodel, determines the homogenic 
virtual surface, which is identical to the geometric surface of the observed object. 

To carry-out the suggested method of stereoscopic measuring image points, the device 
(fig. 3 and fig.4) is offered, containing video-cameras 3 and 4 with infrared color filters 5 and 
6 for recording the eyes movements 7 and 8 accordingly, the video-camera 9 and the mirror 
10 to track the head movements 11, system of video-capture the image by the personal 
computer, the Main Monitor 12 for output of the stereo-image under review, the additional 
(controlling) monitor 13 to visually control the process of observation and to operate the 
observation process, system of stereo-surveillance 14, system of infrared eyes highlighting 
15 and eyeglasses frame 16 with the specially shaped marks, for example, in ellipsoidal 
shape, 17 and 18 (fig.5) 
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Fig.5 represents the eyeglasses frame 16 with the specifically shaped marks, for example, 
in ellipsoidal shape. 17 and 18, the marks 17 are located in the vertical plane in a way to 
make their images to get captured by the corresponding cameras 3 and 4, recording the eyes 
movements 7 and 8 accordingly, and the special marks 1 8 are located on the horizontal plane 
in a way to get their image captured through the mirror 10 by the video-camera 9. The 
images 17 and 18 are used for the head movements tracking. 

The scheme of the location of the main 3 and 4, and the additional video-cameras 19 and 
20 for tracking the eyes movements 7 and 8 in the three-dimensional space is represented on 
the fig 6. 

The device for measuring the spatial characteristic of the object by its stereo-images 
works the following way. 

For observation of the object based on its stereo-images, the stereoscopic images are 
displayed on the screen of the main monitor 12 and the calibration of the system has to be 
done for the certain observer, at the meantime the observer observes static and dynamic test 
objects, displayed on that monitor. The main idea of the calibration is to determine the 
dependencies between the position data of the pupils of the eyes centers, captured by the 
video-cameras 3 and 4 at the moments of sight stabilization during the observation of the test 
objects on the monitor screen 12, and the position data of those objects with the subsequent 
consideration of the psycho-physical particularities of the certain observer at the time of 
observation and analysis of those results. The calibration can be done either in monocular 
regime (both eyes focusing on the mono-image of the test objects on the monitor screen), or 
in stereoscopic regime (focusing on the virtual models of the three-dimensional test objects, 
using the system for stereo-view). 

Directly, the observations has to be done by focusing on the stereoscopic images of the 
observed object with the fixation of the sight trajectory with the consideration of the 
calibration results, detection of the fixation areas and points with the control by the 
condition of coplanarity and the following determination of the spatial position data of the 
object. The determination of the spatial position data of the points of the object surface can 
be done by the analysis of the lengthwise Parallax P with the use of the set of the two- 
dimensional position data of the corresponding points in the fixation areas on the base of 
transformations, which are used in photogrammetry or projective stereometry. The 
construction of a three-dimensional model is done by orientation of the virtual model 
constructed, relative to the set of the fixed basic points, assigning the external system of the 
position data of the object. 

The compensation of the head movements is realizaed by determination of the 
movements factors by the computed movements of the image with the use of the special 
marks 17 and 18 and entry of the corresponding compensating amendments in the position 
data of the pupil of the eye, the camera, tracking the head movements, must be synchronized 
with the video-cameras, tracking the eyes movements. 

The use of the additional video-cameras 19 and 20 for tracking the eyes micro- 
movements allows to determine the three-dimensional position of the pupil of the eye and to 
increase the precision of the sight direction computing. 

Control of the observations is realized by the feedback communication, when the fixation 
areas with the correctly calculated location of the point of intersection of the corresponding 
rays are marked on the screen of the controlling monitor 13 by imprinting the color 
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markers, and on the screen of the main Monitor 12 by changing the color parameters of the 
part of the image, corresponding with that fixation. The feedback makes possible for the 
observer not only to control the progress of work (i.e. to see the areas of the image, in which 
the review is already done), but to estimate the quality of the observation as well, analyzing 
the color of the markers, imprinted into the image on the controlling monitor 13 (the color of 
the markers is determined by the values of divergence of the residual vertical parallaxes, 
calculated with the condition of coplanarity and corresponding with the certain points of 
fixations.) Because the mechanism of feedback shows the areas, where the observations have 
been already done, the control results also can be used at the time of recommencement of 
work after interruption. 

The claimed method and the device of stereoscopic measuring image points can be 
utilized industrially in computer systems, assigned for digital stereoscopic measuring as well 
as in the fields like digital interaction photometry, image detection, three-dimensional 
measuring in medicine, biology, natural sources research, mine workings, natural sources 
workings, assessing risks of the natural and man-caused disasters and the effects thereof, 
interactive teaching systems, systems for stereo-vision tests, system of professional aptitude 
tests, computer and television games. The industrial adaptability of the invention has been 
proved by the tests of the sample of the device, carrying out the claimed method. 



FORMULA OF THE INVENTION 



1. The method of stereoscopic image points measuring, consisting in the construction of the 
stereoscopic model set by the pair of overlapping images, determination of the position 
of the aiming axis of the eyes during stereoscopic perception of that model and recording 
the observation data at the moments of the current eyes fixation, which differs by the 
computing the projection of the area of the sight fixation on the monitor screen with the 
observed objects, for each eye, and selecting the typical points of the object being 
observed. 

2. Method of the p. 1, which differs by the following: the typical points with the same name 
of the observed object, selected on the fragments of the digital stereo-images, 
corresponding with sight fixation areas, must be identified for the left and right eye by 
time synchronization. 

3. Method of any of p. 1-2, which differs by the following: the typical points of the same 
name of the observed object, selected on the fragments of the digital stereo-images, 
corresponding with the eye fixation areas, are identified for the left and right eye, taking 
into consideration the intersection of the corresponding rays, determined by the vectors 
coplanarity equation. 

4. Method of any of p. 1-3, which differs by the following: before starting the observation 
the system calibration has to be done by observation the image with test-objects, with the 
known position data in the system of position data of the main monitor, comparing the 
position data of the pupils of the eyes centers, determined in the system of the position 
data of the camera, with the position data of the position data of the test object pictured 
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on the main monitor and the following selection of the mathematic dependencies, 
describing mutual transformations of the position data. 

5. Method of p.4, which differs by the following: during the system calibration test objects 
are presented for the observation with the different conditions, such as time, duration and 
order of appearance of the test objects, location, size, shape and color of the test objects, 
surrounding background, static or dynamic regime of test objects appearance. 

6. Method of any of the pp. 1-3, which differs by the following: additionally, during the 
observation time the visual control of the measuring is done on the screen of the main 
monitor by imprinting the color markers into the area of image, corresponding to said 
fixations. 

7. Method of p. 6, which differs by the following: the visual control of the measuring is done 
on the screen of the main monitor by modifying the color parameters of the area of the 
observed image, corresponding to said fixations. 

8. Method of any of the pp 1-7, which differs by the following: the compensation of the 
observer's head movements is tracked by calculation of the movement in the position of 
the aiming axis of both eyes by the images of separate parts of the observer's head. 

9. Method of p. 8 which differs by the following: the compensation of the observer's head 
movements is done by tracking the movements of several marks fixed on the observer's 
head. 

10. Method of p.9 which differs by the following: the observer's head movements is tracked 
by the marks, fixed close to the observer's eyes to make the images of those marks to get 
captured by the video-cameras, tracking the observer's eyes movements. 

11. Method of p. 10 which differs by the following: marks for tracking the observer's head 
movements, are specifically shaped, which allows detecting the location and orientation 
of the mark, and, accordingly, the observer's head movements. 

12. Method of p. 8 which differs by the following: the parameters of the head movements are 
detected on two different planes. 

13. Method of any of pp. 1-12, which differs by the following: the position of the pupil of the 
eye during the eyes movements registering has to be determined in the three-dimensional 
space by receiving two images of each eye by two synchronized video-cameras, fixed 
with different foreshortening. 

14. The device for the stereoscopic measuring of the position data of the image points, 
consisting of two video-cameras for tracking the eyes movements, video-camera for 
tracking the head movements, system for video-capture of the image by personal 
computer, monitor for displaying the image, system of stereo-observation, allowing to 
observe stereoscopic images, which differs by containing a construction made in a shape 
of eyeglasses with the specially shaped marks, for example, made in ellipsoidal shape, 
which of those are located in the vertical plane in a way to make the images of the marks 
possible for capturing by the cameras, tracking the eyes movements. 

15. Device of p. 14 which differs by the following: the eyeglasses frame has the specifically 
shaped marks, for example, made in ellipsoidal shape, which are located on the 
horizontal plane, and the mirror fixed above the observer's head is installed additionally 
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into the Device; and the surveillance video-camera for tracking the head movements is 
fixed in a special way to capture at the same time the part of the head and the reflection 
in the eyeglasses frame mirror with the marks placed on it horizontally. 

16. The Device of any of pp. 14-15, which differs by the following: in addition to the main 
two video cameras for tracking the movements of each eye separately, there are two 
additional video cameras installed in to track the movements of each eye synchronically 
by the main and additional video-camera from two points. 

17. The device of any of p.p. 14- 18, which differs by containing an additional monitor to 
control visually the process of observation and to operate the process of observation. 

18. The device of any of pp.14-17, which differs by the following: video-cameras have a 
system of infrared highlighting the eyes area. 

19. The device of any of pp. 14-18, which differs by the following: the video cameras have 
the infrared color filters to cut off the parasite highlighting in the visible range of 
spectrum. 
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